Persistence of Naïve Statistical Reasoning Concerning Analysis of Variance

Two experiments were conducted to assess the intuitive reasoning of students when examining data from an analysis of variance design. Participants were shown hypothetical datasets that differed with regards to within-group and/or between-group variability, and were asked to judge the amount of evidence that each provided in support of a particular theory. The first experiment (n=57) examined the influence of presentation format of the hypothetical datasets. Participants were randomly assigned to receive the hypothetical datasets in one of two formats: (1) group data stacked vertically in a single column, or (2) group data displayed side-by-side in two columns. In the second experiment (n=13), students’ reasoning about the hypothetical datasets was assessed both before and after completing an introductory graduate level statistics course. Consistent with prior research, participants tended to place an inordinate amount of weight on the relative importance of betweengroup, as opposed to within-group, variability. The results indicate that neither presentation format (Experiment 1) nor statistics training (Experiment 2) is enough to overcome this aspect of naïve statistical reasoning.